Evaluation of Criteria for Information Retrieval

We investigate van Rijsbergen’s -measure, break-even point, and 11-point averaged precision, all of which can be translated into 1-dimensional scalar quantity from the precision and the recall. These investigations can be done by comparing to tetrachoric (four-fold) correlation coefficient and phi (four-fold point) coefficient, which are often used as the index of statistical association in a 2 2 contingency table. The results show that when a fallout rate is less than 0.1, (1) the measure has similar properties of the phi coefficient, (2) the break-even point is almost equivalent to a phi coefficient, and (3) the 11-point averaged precision should be a measure which is larger than a phi coefficient and has a value smaller than a tetrachoric correlation coefficient.